I’ve been thinking a lot recently about trust, science and money, although perhaps not in that order. This philosophical interlude was prompted by a blog posting from a venture capitalist in Cambridge about the reproducibility of academic research. His take was that, before you start up a company and start ringing the phones, you should spend some time and money making sure that the new technology behind it actually works (and works again). His estimate, in fact, was that about half the time it won’t.

This is not an accusation of fraud, by the way. Everyone who’s done real edge-of-the-known-world research realizes that there are always a lot more variables in play than you can pin down. It can make for a frustrating experience when the big new idea works, then doesn’t work, then works again, when everything is seemingly being done exactly the same way. You find yourself holding your breath every time you run things, wondering what you’re in for this time. Until some of the conditions get worked out, a new discovery can seem only partially real, materializing in this world for a brief period only to fade out again.

So there doesn’t have to be bad intent for trouble to occur when you try to get some hot new technology to perform. In fact, I suspect that you run into very few outright frauds if you’re running a reputable VC business: what sort of lunatic would try to run the whole startup process with something that he knew was actually a con? It’s probably happened, since there are all sorts of lunatics in this world, but I’d bet that, for the vast amount of the time that everything went sour, it was because of wobbly science and/or self-deception.

That last factor is never to be underestimated. It sneaks up on you; it’s not like you say, “Well, I think I’ll just sort of let these troublesome parts slide and pull the wool over my own eyes for a while.” Not at all – you’re working on that new idea, trying out experiments, and if some of them don’t quite give you the answers you were hoping for, well, that’s science, isn’t it? All those variables, you know (just like we were talking about above), you can’t expect everything to work every time. But there’s an invisible line, one that’s never quite in the same place every time, and it marks the boundary between keeping an open mind and fooling yourself. If you run the same experiments, and most of them still don’t work the “right” way, it’s not that you don’t have reproducible results yet. You do. The more reproducible ones, though, just aren’t the results that you want.

I think, in these cases, that the idea of forming a company can be just what’s needed. That’ll move a discovery into the real world for sure. It’ll have to get up on stage, in front of the potential investors, and hit its marks every time, reciting its lines without any puzzled looks and awkward pauses. If not, well, you now have a strong incentive to figure out what’s wrong. It’s one thing to try to convince the referees from the big journals, but it’s quite another to convince the people with the long green. Money, or the prospect of it, is a great focuser of the mind. If it’s used in a good cause, there’s nothing like it.

But it doesn’t always get used that way, for sure. It’s strong stuff, the thought of gain. And if it’s used in a not-so-good cause, well, there’s nothing like it, either. The prospect of a big success can be a big incentive for self-deception, or worse. You might not see many people start companies by deceit, but there sure have been some that have been kept alive that way.
One of the grey areas comes up when the company is a going concern, but when it’s time to attract the attention of a bigger outfit in order to stay that way. The prospect of a development deal or milestone payment can (at times) focus the mind just a bit too much.

I’m becoming more and more convinced, for example, that we in the industry – especially the small-company end of it – have gotten a little too slick about setting up Phase II trials.
There are just too many Phase III failures where everyone is left standing around in the wreckage, saying, “It worked in Phase II � what happened?” What happened, more than likely, is that the patients were so carefully selected, and the trial’s endpoints so finely tuned, that the drug really didn’t have a chance to survive in the real world (not even that rough approximation of it that it encountered in Phase III). The purpose of some small-company Phase II trials is less to show that their drug is really going to work as to show it in the most favorable light and from the most favorable angle. Just enough to attract a suitor, not enough to cause any potential trouble. It’s a moral hazard, or it can be.

So if the failure rate for new academic discoveries is 50%, how bad is that compared to some of the later-stage stuff? The problem is, that strategy of “get someone else to replicate it” doesn’t work out so well in the clinic. Although in some cases, it really would be cheaper to do a whole new Phase II than to get wiped out in Phase III – that’s one of those thoughts that only occurs to you when it’s too late. I realize that there’s often a Phase IIa and a IIb, but it doesn’t matter how you name them if you’re not going to be honest with the way they’re run. And there you have one of the fundamental problems with our whole industry, the way that the stakes get higher and higher as we go along, while at the same time the opportunities to fix any problems get more and more constrained.

If we’re going to get out of the jam that this business is in, one thing that would help would be finding some way out of that particular box. It isn’t a new thought: people have been trying to “fail cheaper” and “fail faster” for some time now. But most of our improvements seem to have helped out in the earlier phases of development, leaving us with a straighter shot at the Phase III collapses, which probably wasn’t the original plan. We’d be better off, you might think, finding reasons not to run quite as many Phase III trials (or at least not so many losers). Short of that (since I don’t know how to do that!), we could at least try to make sure that the incentives aren’t pointing people in the wrong direction.

And we shouldn’t be shy about spending a little extra time and money making sure that we’re doing the right thing – starting the right company, the right discovery program, or the right clinical trial. Looking foolish now is an excellent substitute for looking even more foolish later.

Derek B. Lowe has been employed since 1989 in pharmaceutical drug discovery in several therapeutic areas. His blog, In the Pipeline, is located at http://www.corante.com/pipeline and is an awfully good read. He can be reached at [email protected].